Skip to content

Handle Windows long file names in a few more places.#1382

Merged
jayconrod merged 1 commit intobazel-contrib:masterfrom
pmuetschard:long
Mar 16, 2018
Merged

Handle Windows long file names in a few more places.#1382
jayconrod merged 1 commit intobazel-contrib:masterfrom
pmuetschard:long

Conversation

@pmuetschard
Copy link
Copy Markdown
Contributor

Go can handle long file names on Windows if they are not relative. This change makes some file paths absolute so that go can handle them. See https://github.com/golang/go/blob/b86e76681366447798c94abb959bb60875bcc856/src/os/path_windows.go#L133.

The protoc binary cannot handle long filenames either, so this change generates go protos in the temporary folder and then moves the contents into place.

Copy link
Copy Markdown
Collaborator

@jayconrod jayconrod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ianthehat has more experience with Windows than I do and should take a second look. This change looks good to me, but I have a few readability comments, and there's a test failure.

So if I understand correctly, in order to be safe on Windows, we need to make every path absolute before calling any file API on Windows? Is this a common problem, or does it affect a few things (protoc) that have particularly long paths?

Comment thread go/tools/builders/compile.go Outdated
if err := goenv.update(); err != nil {
return err
}
absOutput := abs(*output)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without any comment here, we'll probably forget why this is here and remove it in six months. :)

It would also be good to have a more detailed comment on abs itself that references path_windows.go as you did in the commit message.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment thread go/tools/builders/protoc.go Outdated
info.created = true

if f.IsDir() {
if err := os.Mkdir(abs(filepath.Join(*outPath, relPath)), f.Mode()); !os.IsExist(err) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move the abs call to *outPath next to tmpDir above and use the result here. The comment can explain why both are needed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

return err
}
tmpDir = abs(tmpDir)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

defer os.RemoveAll(tmpDir) instead of calling it at the end.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment thread go/tools/builders/protoc.go Outdated
}
files[path] = info

foundInfo := files[relPath]
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if foundInfo, ok := files[relPath]; ok { ...

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

outputs := []*goMetadata{}
for _, input := range inputs {
if m, err := readGoMetadata(bctx, input, true); err != nil {
if m, err := readGoMetadata(bctx, abs(input), true); err != nil {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like tests are failing because //go/builders:filter_test does not include env.go, which it now needs.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Go can handle long file names on Windows if they are not relative.
This change makes some file paths absolute so that go can handle them.
See https://github.com/golang/go/blob/b86e76681366447798c94abb959bb60875bcc856/src/os/path_windows.go#L133.

The protoc binary cannot handle long filenames either, so this change
generates go protos in the temporary folder and then moves the contents
into place.
@pmuetschard
Copy link
Copy Markdown
Contributor Author

pmuetschard commented Mar 15, 2018

So if I understand correctly, in order to be safe on Windows, we need to make every path absolute before calling any file API on Windows?

Yes.

Is this a common problem, or does it affect a few things (protoc) that have particularly long paths?

The path length limitation is a Windows global thing. Most tools work around it, by using the \\?\c:\foo\bar form of paths. Golang does, too, but, as mentioned in the comment of path_windows.go, bails if the input path is relative (the \\?\ form has some processing limitations, including only accepting absolute paths). So, the absolute path vs relative path, is a golang thing. It is indeed more of a problem with protobuf and go with bazel due to paths like this:
bazel-out/x86_windows-fastbuild/bin/external/io_bazel_rules_go/proto/wkt/windows_amd64_stripped/source_context_go_proto~/google.golang.org/genproto/protobuf/source_context/source_context.pb.go. Although as is, this path is just ~200 chars, the length check also has to pass on the abosolute form, which includes the c:\users\user\appdata\local\temp\_bazel_user\xxxxxxxx\execroot\workspace\ prefix...

@jayconrod jayconrod merged commit 2d33362 into bazel-contrib:master Mar 16, 2018
@pmuetschard pmuetschard deleted the long branch March 16, 2018 18:53
jayconrod pushed a commit that referenced this pull request Mar 29, 2018
Go can handle long file names on Windows if they are not relative.
This change makes some file paths absolute so that go can handle them.
See https://github.com/golang/go/blob/b86e76681366447798c94abb959bb60875bcc856/src/os/path_windows.go#L133.

The protoc binary cannot handle long filenames either, so this change
generates go protos in the temporary folder and then moves the contents
into place.
pmuetschard added a commit to pmuetschard/rules_go that referenced this pull request Jan 11, 2021
pmuetschard added a commit to pmuetschard/rules_go that referenced this pull request Jan 11, 2021
yushan26 pushed a commit to yushan26/rules_go that referenced this pull request Jun 16, 2025
…ib#1382)

This adds a changelog in a keepachanglog.com style format.

It's initially populated with currently unreleased behavior and the last
release's (0.24.0) changes.

Work towards bazel-contrib#1361
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants